Summary

Reconcile the Fish Audio streaming websocket port with the superseding Python and JavaScript behavior now on main.

• Reuse /v1/tts/live websocket connections through the existing JS ConnectionPool and expose TTS.prewarm().
• Close pooled sockets from TTS.close() and remove failed or idle-closed sockets before reuse.
• Invalidate the pool when model changes because Fish Audio sends the model in the websocket handshake header.
• Reject an in-flight prewarm that becomes stale during a model change, reconnecting with the current model.
• Preserve immediate first-frame emission and terminal-frame handling from livekit-plugins-fishaudio: drop the startup prebuffer (fish now streams the opening chunks smoothly) agents#6423 and livekit-plugins-fishaudio: drop startup prebuffer #2033.
• Do not restore the obsolete two-chunk startup prebuffer.

Source diff coverage

This reconciles both Python sources and current JS main:

• livekit-plugins-fishaudio: pooled websocket reuse + prebuffer to fix cold-start crackle agents#6368: pooled websocket reuse, prewarming, 300-second session duration, model-change invalidation, and close cleanup.
• livekit-plugins-fishaudio: drop the startup prebuffer (fish now streams the opening chunks smoothly) agents#6423: removes the temporary two-chunk prebuffer and emits the first complete frame immediately.
• livekit-plugins-fishaudio: drop startup prebuffer #2033: JS terminal-frame behavior and immediate first-frame semantics.

JavaScript adaptations:

• Durations use milliseconds (300_000).
• Node ws lifecycle and readyState checks replace aiohttp websocket behavior.
• Shared ConnectionPool.withConnection handles checkout, return, abort, and failure removal.
• A private module-local pool association avoids exposing ConnectionPool as public Fish Audio API.

TDD and verification

RED lifecycle tests initially reported 3 failures:

• An in-flight s2-pro prewarm survived a change to s2.1-pro.
• Failed-socket cleanup needed a separate subsequent-turn assertion matching the zero-retry stream contract.
• Reusing a websocket that closed while idle timed out.

GREEN: plugins/fishaudio/src/tts.test.ts passes all 10 tests, covering sequential reuse, prewarming, completed and in-flight model invalidation, failure and close cleanup, idle closure, immediate first-frame emission, and final-frame handling.

Passed checks:

• pnpm test -- --run plugins/fishaudio/src/tts.test.ts.
• pnpm --filter @livekit/agents-plugin-fishaudio lint.
• pnpm --filter @livekit/agents-plugin-fishaudio build.
• pnpm build.
• pnpm lint.
• pnpm format:check.

The Fish Audio changeset is a patch release.

Cue voice E2E

• Session: sid_f5a682549ea6.
• Exact head: 19abcfd199b0cd58144aa8df5ec0fd130a951086.
• Providers: credentialed LiveKit STT/LLM and credentialed Fish Audio TTS.
• Boundary: temporary protocol-compatible local websocket forwarder counted upgrades, captured model headers, and timestamped provider frames; it was built before use and removed afterward.

Turn 1 waited for full playout with:

agent_state_changed(.new_state="AS_SPEAKING") -> agent_state_changed(.new_state="AS_LISTENING")[30s]

It resolved at session-relative 9513 ms. TTS TTFB was 156.669 ms. Prewarm opened upgrade #1 with model s2-pro; the first turn reused that same connection, with no second upgrade before synthesis.

Turn 2 changed the model and waited for the tool, speech, and full playout:

function_tools_executed(.function_calls[0].name="switchFishModel") -> agent_state_changed(.new_state="AS_SPEAKING")[15s] -> agent_state_changed(.new_state="AS_LISTENING")[30s]

It resolved at session-relative 24214 ms. TTS TTFB was 147.254 ms. Exactly upgrade #2 opened with model s2.1-pro, followed by successful speech and playout.

Immediate-frame evidence:

• The deterministic test resolves stream.next() from the first complete provider audio event before a second provider event is sent.
• On the credentialed turn, the first and second provider frames were 89.579 ms apart and agent speech began from the first frame before the second arrived.
• Worker logs record playout completed without interruption for both turns.
• Cue ended cleanly before artifacts were inspected; both per-turn WAVs contain nonzero agent-channel audio.

Persisted artifacts and SHA-256:

• Full session WAV: ~/.cue-cli/sessions/sid_f5a682549ea6/recording.wav — 5745fdc5272427b8fae907fea11cf0e2e947ccd66bd364a82e6ea71fe1bc2215.
• Turn 1 WAV: ~/.cue-cli/sessions/sid_f5a682549ea6/recordings/001_run/recording.wav — 8e52359e73dd2eb64fb942a85c544bc662f0c04601682ad45639f03c6b31bbd0.
• Turn 2 WAV: ~/.cue-cli/sessions/sid_f5a682549ea6/recordings/002_run/recording.wav — 9bdc45a68f897955a1ffe7d6513e7877babd6c6f8aa409fb62702d1b44a14721.
• Turn 1 result JSON: ~/.cue-cli/sessions/sid_f5a682549ea6/recordings/001_run/result.json — 5be009b38e7c6c91e2f1f02b8f11f6e943bdf2176536065b647af56144bbdb56.
• Turn 2 result JSON: ~/.cue-cli/sessions/sid_f5a682549ea6/recordings/002_run/result.json — 3f98d27ae2985eb88394204fc7881481f97badbc36f7521bdaa9f30f5149de43.
• Worker/proxy log: /tmp/task-2017-final-worker.log — 837b04c57b7a845bcb487cde918fc19de2fb8ae8f374cf846753f7aafd1e5ca0.

Boundary limitation: proxy timestamps are protocol-compatible provider receive/forward timestamps, not internal Fish server timestamps. SDK immediacy is established by the deterministic first-event test and corroborated by Cue TTFB and speech timing.

Review

Independent reconciliation reviews returned:

• Spec compliance: APPROVED.
• Code quality: APPROVED with no Critical/Important findings.

Ported from livekit/agents#6368 and reconciled with livekit/agents#6423 and #2033.